Existing networking and interconnect technologies have failed to keep pace with the development of computer systems, resulting in increased burdens being imposed upon data servers, application processing and enterprise computing. This problem has been exasperated by the popular success of the Internet. A number of computing technologies implemented to meet computing demands (e.g., clustering, fail-safe and 24×7 availability) require increased capacity to move data between processing nodes (e.g., servers), as well as within a processing node between, for example, a Central Processing Unit (CPU) and Input/Output (I/O) devices.
With a view to meeting the above described challenges, a new interconnect technology, called the InfiniBand™, has been proposed for interconnecting processing nodes and I/O nodes to form a System Area Network (SAN). This architecture has been designed to be independent of a host Operating System (OS) and processor platform. The InfiniBand™ Architecture (IBA) is centered around a point-to-point, switched IP fabric whereby end node devices (e.g., inexpensive I/O devices such as a single chip SCSI or Ethernet adapter, or a complex computer system) may be interconnected utilizing a cascade of switch devices. The InfiniBand™ Architecture is defined in the InfiniBand™ Architecture Specification (the IBA specification) Volume 1, Release 1.0, released Oct. 24, 2000 by the InfiniBand Trade Association. The IBA supports a range of applications ranging from back plane interconnect of a single host, to complex system area networks, as illustrated in FIG. 1 (prior art). In a single host environment, each IBA switch fabric may serve as a private I/O interconnect for the host providing connectivity between a CPU and a number of I/O modules. When deployed to support a complex system area network, multiple IBA switch fabrics may be utilized to interconnect numerous hosts and various I/O units.
Within a switch fabric supporting a System Area Network, such as that shown in FIG. 1, there may be a number of devices having multiple input and output ports through which data (e.g., packets) is directed from a source to a destination. Such devices include, for example, switches, routers, repeaters and adapters (exemplary interconnect devices). Where data is processed through a device, it will be appreciated that multiple data transmission requests may compete for resources of the device. For example, where a switching device has multiple input ports and output ports coupled by a crossbar, packets received at multiple input ports of the switching device, and requiring direction to specific outputs ports of the switching device, compete for at least input, output and crossbar resources.
In order to facilitate multiple demands on device resources, an arbitration scheme is typically employed to arbitrate between competing requests for device resources. Requests may include both unicast and multicast transmission requests pertaining to packet received on any one of the multiple input ports of the switching device. Arbitration schemes typically include either (1) distributed arbitration schemes, whereby the arbitration process is distributed among multiple nodes, associated with respective resources, through the device or (2) centralized arbitration schemes whereby arbitration requests for all resources is handled at a central arbiter. An arbitration scheme may further employ one of a number of arbitration policies, including a round robin policy, a first-come-first-serve policy, a shortest message first policy or a priority based policy, to name but a few.
The physical properties of the IBA interconnect technology have been designed to support both module-to-module (board) interconnects (e.g., computer systems that support I/O module add in slots) and chassis-to-chassis interconnects, as to provide to interconnect computer systems, external storage systems, external LAN/WAN access devices. For example, an IBA switch may be employed as interconnect technology within the chassis of a computer system to facilitate communications between devices that constitute the computer system. Similarly, an IBA switched fabric may be employed within a switch, or router, to facilitate network communications between network systems (e.g., processor nodes, storage subsystems, etc.). To this end, FIG. 1 illustrates an exemplary System Area Network (SAN), as provided in the InfiniBand Architecture Specification, showing the interconnection of processor nodes and I/O nodes utilizing the IBA switched fabric.
A number of switching and routing protocols enable the definition of service levels, which may be utilized to identify and differentiate traffic flows within a network. For example, the IBA specification defines a number of service levels (SL) that are utilized to identify different flows within an IBA subnet. The service level associated with a particular packet is carried in the local routing header of a packet and is an indication as to the service class of the relevant packet. While the IBA does not assign specific meaning to each service level, other protocols may do so. Service levels are typically intended to facilitate a mechanism to provide differentiated services, improve switched fabric utilization, and to avoid deadlock.
A number of switching and routing protocols enable the definition of a number of data streams that may be received at, or communicated from, a network (or interconnect) device. For example, the IBA specification defines so-called virtual lanes (VLs). Utilizing the IBA as an example, as a packet is routed across a network (or a subnet), it may be desirable or necessary for that packet to be transferred from one data stream (or virtual lane) to another. Referring to FIG. 2 (prior art), a network 12 including a number of interconnect devices 13 is illustrated. FIG. 2 also illustrates that a certain number of virtual lanes are defined on links between various interconnect devices. It will be noted that links feeding into and out of an interconnect device 13 at the center of the network 12 provide a larger number of virtual lanes, while links feeding into and out of interconnect devices 13 at the edges of the network 12 support a lesser number of virtual lanes. The network 12 may be so implemented as there is a higher probability of link contention towards the center of the network 12. A larger number of virtual lanes are accordingly implemented towards the center of the network 12 to reduce the negative impact of link contention. It will be appreciated that, as a packet travels towards or from the center of the network 12 illustrated FIG. 2, it may be necessary to transfer a particular packet from one virtual lane to another. For example, a packet may be transferred from one virtual lane to another if a particular link does not support a virtual lane previously utilized by the packet.
Again taking the IBA as an example, in order to facilitate the transfer of a packet from one virtual lane to another, the IBA (pages 152-153) specifies a service level-to-virtual lane mapping scheme that may be utilized to transfer a packet from one virtual lane to another as the packet traverses a network (e.g., a subnet). Specifically, service level-to-virtual lane mapping may be required in channel adapters, switches, and routers that support more than one data virtual lanes. The IBA specifies that such service level-to-virtual lane mapping be performed utilizing a programmable mapping table, termed the SL-to-VL MappingTable. An example of this table is provided immediately below:
TABLE 1LengthOffsetComponentAccess(bits)(bits)DescriptionSLV0toVLRW40Then number of the VLon which packets using SL0are output. 15 forces thepackets to be dropped.SL1toVLRW40The VL associated with SL1SL2toVLRW44The VL associated with SL2SL3toVLRW48The VL associated with SL3SL4toVLRW412The VL associated with SL4SL5toVLRW416The VL associated with SL5SL6toVLRW420The VL associated with SL6SL7toVLRW424The VL associated with SL7SL8toVLRW428The VL associated with SL8SL9toVLRW432The VL associated with SL9SL10toVLRW436The VL associated with SL10SL11toVLRW440The VL associated with SL11SL12toVLRW444The VL associated with SL12SL13toVLRW448The VL associated with SL13SL14toVLRW452The VL associated with SL14SL15toVLRW456The VL associated with SL15
Specifically, in the case of an interconnect device in the form of a channel adapter and router, the above table provides a mapping of the service level to a virtual lane supported by an output port of the relevant interconnect device. The table is 16 entries deep, with each port of the relevant interconnect device having an independent table. All 16 possible values for a service level are included within the table. The table indicates the virtual lane number to be used when a packet is transmitted from a particular output port.
In the case of an interconnect device in the form of a switch, the above table maps a service level, input port and output port of the relevant packet to a virtual lane to be used for a next hop within the network.
In short, the above table can be conceptually viewed as a set of tables, one for each output port. Each of these “per output port” tables indicates which virtual lane should be utilized by an outgoing packet, based on a service level associated with the packet and the port of the interconnect device on which the packet arrived.